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Introduction 


The global community has made a commitment to 
high-quality education for every child and young 
person. By the terms of the fourth Sustainable De- 
velopment Goal (SDG), target 4.1, every girl and boy 
should complete primary and secondary education 
by 2030. This education should be freely available to 
all and should lead to “relevant and effective learning 
outcomes” (United Nations Statistics Division, 2017). 


Even if this target is achieved, however, the education 
delivered may be narrow. There is risk of a focus on a 
subset of learning that young people need—especial- 
ly literacy and numeracy—that may fail to address the 
complete set of skills needed to thrive in a changing 
world, such as collaboration, problem solving, critical 
thinking, and creativity. 


Historically, school curricula have been organized 

by discipline, prioritizing the acquisition of knowl- 
edge in discrete subject areas, particularly language, 
mathematics, and science. This approach to school 
learning is entrenched in the education sector: 
textbooks, school organization, teacher education, 
and assessment, all reflect this discrete, disciplinary 


basis. Despite some exploration of project learning, 
especially in primary schools, the tradition of separate 
subject learning, remains current. A recent report has 
documented how many students continue to learn 

in traditional school environments, where they sit at 
desks, listen passively, and memorize fixed bodies 

of knowledge (Care, Kim, Anderson, & Gustafs- 
son-Wright, 2017). This report notes also that teachers 
often receive little professional support to deliver a 
balanced curriculum. 


This tradition is reinforced by assessment and exam- 
inations, both as part of daily routines and high stakes. 
When students need to pass examinations in order to 
progress through the education system, their focus— 
and that of their teachers—will likely be on the content 
of those examinations, to the exclusion of broader 
learning. Therefore, to the extent that examinations fo- 
cus on traditional subjects, and not on social respon- 
sibility, collaborative work, or the arts, they reinforce 
the traditional focus of the curriculum. 


What if, in addition to evaluating an education system 
on the learning outcomes demonstrated by students, 


evaluation also targeted the opportunities students 
have to learn a broad range of skills? With this dual 
approach, looking at both opportunities and out- 
comes, education systems might be able to achieve 
the development goals without narrow delivery. The 
Breadth of Learning Opportunities (BOLO) initiative 
seeks to contribute to this approach by providing tools 
to document 1) whether opportunities are provided 
for learning across a diverse group of domains, and 
2) how the key dimensions of an education system 
(curriculum, assessments, teacher supports, monitor- 
ing, and school resources) align to support delivery of 
breadth of learning opportunities. 


This report describes an initiative undertaken from 
2016-2017 by the Center for Universal Education 
(CUE) at the Brookings Institution and Education Inter- 
national (El) to develop tools to measure the breadth 
of learning opportunities to which children and youth 
are exposed in an education system. First, we provide 
the rationale for developing these tools. Next, we de- 
scribe the tools developed through this initiative. We 
then explain how the tools can be adapted, and some 
of the questions the tools might be used to answer. 
This report is part of a package of documents, which 
includes the most recent version of the tools, a de- 
scription of how the tools were developed, and case 
studies of tool piloting in Kenya and Mexico. 


The importance of a broad set 
of learning opportunities 


The disciplinary basis of contemporary schooling may 
have served well in the past but, as a means of struc- 
turing students’ learning, it is no longer fit for purpose: 
it does not match the dynamic knowledge environ- 
ment already surrounding us, and it is not optimal for 
equipping young people for their adult lives. 


Whether focusing on civil society or the workplace, 
substantial changes can be anticipated in living and 
working arrangements over the coming decades. 


DV 


We know already that teamwork and creativity, for 
instance, are more highly valued in the workforce than 
they are currently emphasized in schooling. A flexible 
skill set is increasingly necessary for success in life 
and work. The International Commission on Financing 
Global Education Opportunity (The Education Com- 
mission) predicts that half of the world’s four billion 
jobs will be at risk due to automation by 2030 (The 
Education Commission, 2016). The increasing com- 
plexity of technology and the rise of globalization have 
ushered in the Information Age, and content knowl- 
edge will no longer be enough for students to suc- 
ceed; they need to be equipped with the breadth of 
skills required to adapt to societal change and thrive 
in the 21st century. 


More critically, there is much that we do not know 
about the future: many of the jobs that today’s school- 
children will be doing have not been created yet, and 
there is little point in preparing them for jobs that may 
not exist in the future (The Education Commission, 
2016). Therefore, the most effective preparation we 
can give young people is one that develops their per- 
sonal capacities and equips them with a broad set of 
flexible competences. It will be for them to respond to 
challenges as yet unknown and create new futures. In 
this new world, the advantage will be with well-round- 
ed individuals who are able to apply their acquired 
knowledge and skills to new and different situations. 


The conceptual model 


The curriculum can be seen from the perspective of 
the student, teacher, school, or system. In the BOLO 
work, we focus on the teacher, school, and system. 
This work sits within the broader education model, 
which is described below. 


The key question for a young person is: what have | 
learned during my time in school? Learning is tak- 
en here to encompass knowledge, skills, attitudes, 
and behavior. This is the curriculum as internalized. 
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Ideally, a comprehensive portfolio covering examina- 
tion and test results, behavioral indicators, self-as- 
sessments, project work, and measures of workplace 
readiness would measure this. This broad approach 
to measurement is necessary because young people 
express what they have learned in individual ways, 
and key learning outcomes are not always pre-deter- 
mined. 


This learning could derive from a range of sources, 
including school. For some young people the school 
may be a minor or even negligible source of learning 
but for the purposes of this project, we consider the 
subset of learning experiences that schools provide. 
This is the curriculum as experienced, sometimes 
described in terms of opportunity to learn. These 
experiences provide coherence, continuity, and links 
between different areas of learning throughout a learn- 
er’s life. 


Examining the experienced curriculum sheds light on 
the situation of many students whose learning patterns 
differ from the norm. Teaching directed at the average 
student is likely to constitute different learning experi- 
ences for highly able students, as well as for students 
with learning difficulties. Information on the curriculum 
as experienced should come primarily from students, 
by questionnaire, focus group, and individual inter- 
view, though it can be supplemented by observation 
of both classroom activities and non-classroom inter- 
actions. 


A major determinant of students’ learning experiences 
is what they are presented with, referring to the curric- 
ulum as implemented. This includes what students are 
taught in lessons, and opportunities for independent 
learning through homework, collaboration with peers, 
structured access to information technology, partici- 
pation in the life of the school, and social interactions. 
This can be measured by examining teacher plans 
and lesson delivery and gathering information on 
homework, use of information technology, opportuni- 


ties for collaborative working, social interactions, and 
so on. 


Teachers operate within a curriculum, assessment, 
and organizational framework set by their school, 
which is the curriculum as planned. This sets out the 
school’s vision for student learning: how it will be real- 
ized in terms of learning and participation opportuni- 
ties; how achievement and progression will be mea- 
sured; and what forms of academic organization the 
school will use. Information on this can be gathered 
by questionnaire, the study of school documentation, 
and the observation of school practices. 


Schools, in turn, operate within frameworks laid down 
by national authorities, usually a national curriculum 
body. These set out the national curriculum, or the 
curriculum as intended, specifying in greater or lesser 
detail what students should learn at different grades 
and how their achievement should be assessed and 
certificated. Information on this can be obtained by 
questionnaire or by scrutiny of documents which are 
normally in the public domain. 


In many countries, there is an intermediate level 
between the intended or official curriculum and the 
planned curriculum—the examined curriculum. This 
is because of the impact of assessment practices, 
particularly high-stakes examinations. The official 
curriculum may well set out a broad view of learning 
but, commonly, examinations restrict themselves to 
a restricted set of learning outcomes, which are then 
prioritized in schools’ curricula. 


These different levels of the curriculum are summa- 
rized in Table 1. 


What do breadth of learning opportunities 
across the curriculum look like? 


Concerns over the narrowness of student learning are 
not new, and there have been numerous attempts to 


Levels of the curriculum and corresponding measurements 


Designation 


Internalized What students have learned 


Experienced Students’ opportunity to learn 


What and how teachers teach, 
use of homework and indepen- 
dent learning, group work 


Implemented 


Planned The school’s vision for learning 


and teaching, assessment prac- 
tices, academic organization 


Examined The learning deemed import- 


ant for testing and examination 
purposes 


Intended National (or other governmental 


unit) curriculum as defined by 
authorities 


set out what a broad-based curriculum would look 
like. One of the best known is the United Nations 
Educational, Scientific and Cultural Organization (UN- 
ESCO) Report Learning: The Treasure Within (UNE- 
SCO, 1996). This put forward four pillars, which were 
deemed the foundations of education: 


e Learning to know. Broad general knowledge with 
in-depth understanding of a few areas, but also 
learning to learn so as to facilitate lifelong learning 


e Learning to do. Occupational skills but also the 
competences to function in many different situa- 
tions, both social and work-related 


e Learning to live together. Understanding other 
people, being respectful of divergent perspec- 
tives and being able to manage conflict 


Level of 
measurement 


How it is measured 


Individual student Analysis of student work, assessment 
data, attitude scales, behavioral 


indicators, workplace readiness 


Teacher/classroom 
Individual student 


Student surveys, observation of 
school practice (e.g. tools that 
measure student experiences as 
opposed to what the teacher does) 


School School surveys and inspections, 

Teacher/classroom analysis of lesson plans, lesson 
observation 

School Analysis of school syllabi, school and 
classroom assessments, academic 
organization 

Country or Analysis of assessment guidelines 

jurisdiction and test papers, and any assign- 
ments, projects or other activities 
considered in final grades or marks. 

Country or Analysis of national, provincial or 

jurisdiction state curriculum statements 


e Learning to be. Developing one’s own personality 
and growing in autonomy, judgement, and per- 
sonal responsibility 


As discussed by Care and colleagues (2017), there 
has been growing awareness in recent years of the 
importance of a broad range of skills, and a shift in 
focus from academic, vocational, and technical skills 
toward an aspiration for education that informs both 
work and life more generally (Pellegrino & Hilton, 
2012; Brewer, 2013; Lippman, Ryberg, & Moore, 
2015). 


There have also been efforts by national governments 
to expand national curricula and education delivery to 
include a broader set of learning domains. In a recent 
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study of national curriculum and policy statements 

in 152 countries, most (76 percent) were found to 
include 21st century skills somewhere in their poli- 
cy documents, and nearly half included skills such 
as communication, creativity, critical thinking, and 
problem solving in their curriculum statements (Care, 
Anderson, & Kim, 2016; Roth, Kim, & Care, 2017). 
However, when it comes to the learning outcomes 
actually assessed in an education system, the focus 
is primarily on reading, mathematics, and, to a lesser 
degree, science (Cheng & Omoeva, 2014). 


Where did the breadth of learning 
opportunities measurement idea originate? 


The work outlined in this report takes its point of de- 
parture from the Learning Metrics Task Force (LMTF). 
This was an initiative convened jointly from 2012-2015 
by the Center for Universal Education at the Brook- 
ings Institution and the UNESCO Institute for Statistics 


LMTF Seven Domains of Learning 


(UIS). The Task Force located its work specifically 
within the post-2015 global development agenda. 

Its purpose was to shift the global conversation on 
education from a focus on access to access plus 
learning. Within that framework, it sought to establish 
a consensus around the knowledge and domains of 
learning deemed important to equip all children and 
young people for a productive, fulfilled life. Following 
a process of review, input from experts and wide- 
spread consultation (in 118 countries), the Task Force 
published its proposals on the future of learning in 
the report What Every Child Should Learn (Learning 
Metrics Task Force, 2013). 


At the heart of this report lie the Seven Domains of 
Learning, as set out in Figure 1. 


Literacy & 
communication 


Culture & 
the arts 


Learning 
approaches & 
cognition 


Social & - Numeracy & 
emotional Primary mathematics 
Early 
Childhood 
Physical Science & 
well-being technology 


bDvV- 


The BOLO Tools: 


Measuring Breadth of Learning 
Opportunities across the 


Curriculum Levels 


While measuring learning outcomes across all sev- 
en domains may not be feasible or desirable, it is 
possible to focus on students’ breadth of learning 
opportunities: establish how their learning experienc- 
es match up with an agreed framework (in this case, 
the LMTF Seven Domains of Learning); identify gaps 
and repetition; and take remedial action as neces- 
sary. Analysis of breadth of learning opportunities is 
one way to review curriculum and identify if it meets 
country aspirations. While this project used the LMTF 
Seven Domains of Learning framework as its definition 
of breadth, the process can be generalized to other 
frameworks as relevant for the users of the tools. 


The BOLO tools bring together the conceptual mod- 
el of curriculum described above and a generally 
agreed-upon framework of broad learning, in this 
case, the LMTF Seven Domains of Learning. In an 
ideal world, students’ actual learning—the internalized 
curriculum—would be the starting point. Gathering 
the requisite information would be extremely labor-in- 
tensive, however, and would require considerable re- 
sources. In addition, student factors would complicate 
this approach. This project has therefore focused on 
students’ learning opportunities, as opposed to their 


achievement, leading to an emphasis on the curricu- 
lum as intended, examined, planned, and implement- 
ed. 


For each level of the curriculum, the official policies 
and, when relevant, the practices of schools and 
teachers are mapped onto the LMTF Seven Domains 
of Learning. The three tools capture the perspectives 
of policymakers, school administrators, and teachers 
to determine where there are differences among these 
three groups. The triangulation capacity of the tools 

is an important feature as it can shed light on how 
different stakeholder groups may experience or per- 
ceive the education system differently. The data can 
be analyzed to determine to what extent the curricu- 
lum embodies breadth of learning opportunities, and 
each curricular subject can be examined to determine 
the barriers and bottlenecks to implementation in the 
classroom. The resulting “map” of domains across 
the levels of curriculum can be used as a basis for 
discourse among education stakeholders within a 
country. 


Three sets of tools examine breadth of learning oppor- 
tunities at the policy, school, and teacher levels, which 
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BOLO theory of change 


maps onto the intended, planned, and implemented 
components of the curriculum. Accordingly, the inten- 
tion at the policy level may be reflected in what is de- 
livered at the school and classroom levels. The tools 
are designed to be adaptable to country context and 
needs. Adaptation is discussed further on page 11. 
The full details of how the tools were developed and 
piloted are available in the technical paper Breadth of 
Learning Opportunities: Technical Report 2018. Table 
2 describes the BOLO instruments, respondents, 
curriculum levels, and countries where the tools were 
piloted. 


An important aspect of the BOLO tools is that they 
are aligned to capture and triangulate the policy, 
school, and teacher perspectives. This alignment was 
achieved by first developing a framework for the poli- 
cy tools with an international group of researchers and 
policy makers. That framework was used as a starting 
point for drafting the teacher tool through workshops 
and piloting with teachers in Kenya and Zambia. 
Survey development experts used these versions to 
refine the teacher tool and draft the school tool. Final- 
ly, the three tools were piloted together in Kenya and 
Mexico with policymakers, school administrators, and 
teachers. 


The three tools are described below. 


Policies and A broader set 
practices are of learning 
modified to opportunities are 


encourage greater 
breadth of learning 
opportunities 


experienced by 
children and youth 


— 


Policy tool 


An international group of experts in curriculum, 
assessment, and education policy developed the 
policy tool. It provides a framework to review curricu- 
lum and assessment documents at the jurisdictional 
level, which is defined as the administrative level 
responsible for setting policies on curriculum. The tool 
examines the breadth of learning opportunities for 
students set out at the official level, in one focus grade 
for each educational level studied. In some countries, 
the curriculum documents might be developed and 
sourced at a national level, and in others at a sub-na- 
tional level; the policy tool is designed to be adapt- 
able to either situation. The current tool examines the 
extent to which the curriculum covers the LMTF Seven 
Domains, and how a breadth of learning opportuni- 
ties approach is incorporated throughout monitoring, 
assessment, teacher training, and resource provi- 
sion. The policy tool is designed to be completed 

by an inter-agency team including government and 
non-government stakeholders. The tool contains five 
components: 


1. User Information: Information about the lead 
respondent and the names, positions, and affil- 
iations of the committee or team members who 
completed the tool. 


DE? 


BOLO tools and piloting 


Curriculum Countries piloted to | Education levels 
R ndent : 
geponde level date piloted to date 
Policy tool Policymaker Intended, exam- Argentina, El Salvador, —Pre-primary, primary, 
ined Kenya, Mexico, Pales- lower secondary 


tine, Pakistan, Rwanda, 
Tunisia, and Zambia 


School tool School administrator Planned Kenya, Mexico Primary, lower secondary 
Teacher tool Teacher Implemented Kenya, Mexico, Zambia Primary, lower secondary 
Policy, school, Policy makers, Intended, exam- Kenya, Mexico Primary, lower secondary 
and teacher tools = school administra- ined, planned, 

(alignment pilot) tors, teachers implemented 

2. System Information: An overview of the ed- there is a major or minor focus on the subdomain 


ucation system and the curriculum using the 
International Standard Classification of Education 
(ISCED) (UNESCO Institute for Statistics, 2011) 
to determine the levels of education on which the 
remainder of the tool will focus. This section also 
includes questions about the length of the school 
day, week and year, mandated class sizes, and 
general requirements for teachers. 


. Subjects and Timetable: For each grade level, a 
list of the mandatory and optional subjects in the 
curriculum, and the hours per week scheduled 
according to the curriculum policy. 


. Subject Background — Curriculum Policies for 
Mandatory Subjects: For each mandatory sub- 
ject, a series of questions on the policies related 
to monitoring, examinations and assessments, 
and teacher training and professional develop- 
ment. 


. LMTF Domains Mapping: A matrix of the LMTF 
subdomains mapped to the subjects in the na- 
tional curriculum, with boxes to check on whether 


in each curricular subject. This matrix is complet- 
ed for the focus grade for each education level 
examined in the study. 


School tool 


The school tool was developed by CUE with support 
from FHI 360, in order to align the policy and teacher 
tools. It is intended to examine the breadth of learning 
opportunities for students as evidenced by school-lev- 
el curriculum policies, available facilities, and govern- 
ment oversight of school instruction. The school tool 

is intended for completion by school administrators 
(principals, head teachers, etc.). The school tool con- 
tains five components: 


1. User and School Information: Information about 
the school type, location, management, grades, 
socio-economic status of students, and facilities. 


2. Grade Information: For the grade level of focus, 
information on characteristics of teachers and 
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learners. This section also includes questions 
about the length of the school day, week and 
year, class sizes, and general qualifications for 
teachers. 


3. Subjects and Timetable: For each grade level, a 
list of the mandatory and optional subjects in the 
curriculum, and the hours per week scheduled 
according to the school’s policy. 


4. Information on Mandatory Subjects: For each 
mandatory subject, a series of questions on the 
government and the school’s policies and prac- 
tices related to monitoring, examinations and as- 
sessments, and teacher training and professional 
development. 


5. LMTF Domains Mapping: A matrix of the LMTF 
subdomains mapped to the subjects in the 
school’s curriculum, with boxes to check on 
whether there is a major or minor focus on the 
subdomain in each curricular subject. This matrix 
is completed for the focus grade. 


Teacher tool 


The teacher tool was developed by El with teacher 
unions in Kenya and Zambia to examine breadth of 
learning at the classroom and teacher levels (Kenya 
National Union of Teachers & Education International, 
2017; Zambia National Union of Teachers & Education 
International, 2017). It was refined to align with the 
school and policy tools by CUE and FHI 360. In addi- 
tion to gathering information on what is actually taught 
in schools and time allocations to the different do- 
mains, the tools survey the pre-service and in-service 
education received, time spent on various activities 
across the learning domains, types of assessments 
used, and resources available. The teacher tool con- 
tains six components: 


1. Background Information: Basic information 
about the enumerator, country, grade level, and 
school. 


2. Teacher Information: The teacher’s education 
level, grades taught, gender, specialty, and 
school facilities to which the teacher has access. 


3. Class information: Information about learners 
and class schedule. 


4. Subjects and Timetable: For each grade level, 
a list of the mandatory and optional subjects the 
teacher teaches, and the hours per week schea- 
uled according to the school’s policy. 


5. Information on Mandatory Subjects: For each 
mandatory subject, a series of questions on the 
government policies and teacher practices related 
to monitoring, examinations and assessments, 
teacher education and training, and support of- 
fered to teachers to deliver the curriculum. 


6. LMTF Domains Mapping: A matrix of the LMTF 
subdomains mapped to the subjects taught by the 
teacher, with boxes to check on whether there is 
a major or minor focus on the subdomain in each 
curricular subject. This matrix is completed for the 
focus grade. 


Putting the tools together 


The policy, school, and teacher tools contain similar 
questions to allow triangulation of data from the three 
sources. They are meant to be administered together, 
which can be done in multiple ways. In the alignment 
pilot in Kenya, for example, the policy tool was com- 
pleted at a workshop of government officials, teacher 
union officials, and civil society organizations. Then 

a group of policymakers and a representative from 
the Kenya National Union of Teachers (KNUT) ad- 
ministered the school and teacher tools in a random 
sample of schools. This has the advantage of increas- 
ing government buy-in for the results, but as reported 
by stakeholders in Kenya, can also lead to biased re- 


sponses in favor of what the school administrators and 
teachers think the government officials want to hear. In 
Mexico, one policymaker in each state completed the 
policy tool, and a team of independent researchers 
administered the school and teacher tools. This had 
the advantage of potentially reducing bias, but the re- 
search team also encountered some difficulty access- 
ing schools and teachers, and the individual policy- 
makers were not always aware of the current policies. 
During the piloting of the teacher tools in Kenya and 
Zambia, teachers were trained as enumerators, which 
facilitated access to the schools and teachers, but in 
some cases, the tools were not completed adequate- 
ly. There are advantages and disadvantages to each 
approach, and they need to be considered along with 
the purpose and intended use of the tools. 


Before administering the tools, a small pilot should be 
undertaken to check for relevance and understanding 
among participants. The data can be triangulated 
such that a policy or research team should ask fol- 
low-up questions of participants to determine whether 
misalignment between policy, school, and teacher 
surveys is real or due to participants not interpreting 
the questions in the same way. Rubrics concerning 
how a “major” or “minor” emphasis in the curriculum is 
defined may need to be developed in each jurisdiction 
before a full administration of the tools. The next sec- 
tion provides more guidelines on adapting the tools. 


The results of the tools can be put together in a data 
visualization, or “heat map”, which shows the degree 
to which the LMTF Seven Domains of Learning are 
emphasized within the curricular subjects, according 
to policy, school administrators, and teachers. For 
example, the results from policymakers, school admin- 
istrators, and teachers on how the LMTF subdomains 
within literacy and communication are emphasized 
within the national curriculum subject of English could 
be examined as shown below in Figure 3, with darker 
colors signifying a major emphasis and lighter colors 
signifying little to no emphasis. In this example, the 
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policy makers (denoted by ‘P’) noted that topics such 
as oral fluency, oral comprehension, and reading 
fluency are a major part of the curriculum for English, 
and school administrators (denoted by ‘S’) and teach- 
ers (denoted by 'T’) also said these subdomains were 
emphasized in the English subject. However, teach- 
ers, and to a lesser extent, school administrators, said 
that they emphasize the literacy and communication 
domain in Social Studies and Science, but the cur- 
riculum policy does not appear to emphasize these 
subjects in Social Studies and Science. 


Sample heat map of three LMTF domains and three 
curricular subjects 
Kenya — ISCED 1 —-LMTF Domains 


P: Policy makers 
S: School administrators 
T: Teachers 


Social Science 


Studies 


English 


Pass) TP esi te (Psat 


Number concepts & operations 


Geometry & patterns 


Numeracy & 
mathematics 


Mathematics application 


Oral fluency 


Oral comprehension 


Reading fluency 


Reading comprehension 


Receptive vocabulary 


Literacy & communication 


Expressive vocabulary 


Written expression/composition 


Creative arts 


Culture & 
arts 


Cultural knowledge 


A heat map can also be generated to show the 
degree to which the country or jurisdiction’s curric- 
ular subjects are emphasized throughout the levels 

of the system (intended, examined, planned, and 
implemented). Case studies from Kenya and Mexico 
(forthcoming) describe the process of developing heat 
maps and engaging the study participants in dialogue 
using these tools. 
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Adapting the 


Tools 


The BOLO tools are intended to be adapted to the 
context in which they are to be used. Variations can 
be made to the tools in multiple ways. 


Education levels and types of 
education assessed 


Pre-primary, primary, and lower secondary are 

the schooling levels for which questionnaires were 
developed for this initiative, but upper secondary 
education, technical and vocational education, infant 
and toddler programs, and non-formal education 
programs could also be examined using these tools. 
Programs for students with special needs or learning 
disabilities could also be examined, either as part 

of the traditional education system or separately. To 
adapt the tools to other education contexts, users 
could identify existing frameworks at the national and 
international levels to define breadth. 


Stakeholders participating in the study 


For maximum stakeholder engagement, a study 
should bring together teacher organizations or 


unions along with all government agencies involved 
in developing and implementing policies for teach- 
ing, Curriculum, and assessment. The results can be 
organized to map the perspectives of policymakers, 
administrators and teachers side-by-side to facilitate 
a dialogue between the various stakeholders in the 
system. Additional stakeholder levels (Such as school 
district supervisors) could be included in the study, if 
relevant. 


Learning domains included 


The LMTF framework provides a common framework 
for defining breadth of learning. However, some 
countries have developed their own frameworks that 
incorporate 21st century skills, global citizenship, so- 
cial and emotional skills, and other aspects of a broad 
education. Introducing multiple frameworks could be 
confusing for school personnel in countries where oth- 
er frameworks are being introduced, as was demon- 
strated in the pilot studies in Kenya and Mexico. 


In countries where a new curriculum is being intro- 
duced and the government is interested in learning 


the extent to which the content of the new curriculum 
is already being covered at a school level, the focus 
areas of the new curriculum could be used for map- 
ping, rather than the LMTF domains. In a country that 
has not yet embarked on a curriculum reform, using 
a global framework like the LMTF domains might be a 
useful exercise, and the resulting information on how 
the LMTF domains are represented in the existing 
curriculum could be used to inform the development 
of a new curriculum. 


Jurisdictional level 


In a centrally-administered education system, the 
national government makes policies and they are 
expected to be carried out in each region, so adminis- 
tering one policy tool and triangulating the results from 
the school and teacher tools to this questionnaire are 
possible. The data can also be disaggregated at the 
sub-national level using the same policy tool data in 
each region. 


In a state-administered education system, a separate 
policy tool may be completed for each state. Using 

a common framework of learning domains, such as 
the LMTF framework or the national curriculum and 
standards, is recommended as it enables comparison 
across states. 


Policy questions asked 


For the policy, school, and teacher tools, the ques- 
tions can be modified to meet the needs of the re- 
search and policy interests determined by the country. 
For example, more questions on teacher professional 
development can be included if a country is planning 
to use the results to inform teacher education. 


Aligning with additional data sources 


The BOLO tools could be aligned with previously 
published classroom observation tools to examine the 
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experienced curriculum. While developing and vali- 
dating a classroom observation tool were outside the 
scope of this study, an existing tool that measures the 
classroom environment and student experience could 
be used and the BOLO policy, school, and teacher 
tools could be aligned with the domains included 
therein. Student logs could also be used as potential 
data collection tools aligned with the BOLO tools. 
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Using the 
BOLO tools 


Together, the BOLO tools use multiple sources of 
evidence to gather information on: 


¢ Curriculum documents 

e Assessment and examination frameworks 

e Teacher pre-service and in-service preparation 
e Learning materials 

e Classroom experiences 


The results from the three sets of tools can be com- 
bined to generate a picture of the breadth of learning 
opportunities in the education system at the national 
and classroom levels. Education ministries and other 
actors can use this information to inform curriculum 
reform or review efforts or to examine the implementa- 
tion of the existing curriculum. 


These tools are not accountability mechanisms. They 
are designed to shed light on relevant issues and 
pose specific questions to explore and to inform 
action, particularly by national officials, on how the 
system is operating and where there is and is not 
alignment. By reviewing the strengths and challenges 
in the system, from the national to local levels, this 


information can help to fine-tune policies that seek to 
provide broad learning opportunities. 

The tools are intended to contribute toward improv- 
ing learning outcomes in a wide array of domains by 
giving educators a means to analyze their education 
system. Ultimately, the tools should support gov- 
ernment interventions that increase the breadth and 
depth of learning in their constituencies. Examples of 
questions that the BOLO tools are intended to answer 
are described below. 


1. Does the jurisdictional curriculum offer 
breadth of learning opportunities? 


This question examines to what extent the LMTF Sev- 
en Domains of Learning and subdomains are repre- 
sented in the country’s (or jurisdiction’s) mandatory 
curricular subjects. This information is collected in the 
LMTF domains and subdomains matrix of the policy 
tool. It asks if each LMTF subdomain is covered in the 
curriculum content for each subject, and, if so, wheth- 
er it has a major or minor focus. The policy tool also 
asks the hours per week for which each mandatory 


and optional subject is scheduled according to policy. 
Taken together, the mapping and scheduling data can 
be used to describe how the LMTF Seven Domains of 
Learning map onto the subjects in the curriculum to 
establish whether breadth exists and to what extent. 


2. Are students offered breadth of learning 
opportunities by their schools and teachers? 


This question examines to what extent school admin- 
istrators and teachers report that the LMTF Seven 
Domains of Learning are featured in their school and 
classroom curricula. This information is collected in 
the LMTF domains and subdomains matrix of the 
school curriculum and teacher curriculum question- 
naires. Similar to the policy survey, teachers and 
school administrators are asked if each LMTF sub- 
domain is covered in the curriculum content for each 
subject, and, if so, whether it has a major or minor 
focus. The school and teacher tools also ask the num- 
ber of hours per week each mandatory and optional 
subject is taught. This information can be aggregated 
to examine the extent to which schools and teachers 
focus on teaching across the LMTF Seven Domains of 
Learning and the subjects of the curriculum. 


3. Are the education system 
components aligned to support 
breadth of learning opportunities? 


The sections of the tools that include detailed infor- 
mation on mandatory subjects can be analyzed to 
answer this question. BOLO data can be organized to 
view how the curricular subjects are supported at the 
policy, school, and teacher level on six aspects: 


© Curriculum 

e¢ Monitoring 

e Examinations 

e Assessments 

e Teaching materials 

e Professional development opportunities 
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If a subject is included in the curriculum policy, it 
would be expected that this subject would be mon- 
itored by the school inspection agencies; appear in 
national examinations, classroom assessments, and 
any other type of learning assessment; be part of the 
teaching materials provided to schools (where this 
happens); and be included in professional develop- 
ment opportunities. The data from the BOLO tools can 
be organized to describe how each curricular subject 
is supported, or not supported, both in policy and in 
practice. 


4. Are there discrepancies in 
stakeholder interpretations? 


The BOLO tools are aligned so that the respons- 

es from policy makers, school administrators, and 
teachers can be compared to determine where there 
are discrepancies in interpretations of the curriculum 
across the various groups. Presenting the data from 
all three stakeholder groups together can initiate a di- 
alogue among the groups about whether the discrep- 
ancies are real, and what can be done about them. 


Tool availability 


The BOLO tools, along with a description of how the 
tools were developed and case studies from Kenya 
and Mexico, are available at www.brookings.edu/ 
research/breadth-of-learning-opportunities 
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